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Nowadays an educational mobile apphcation has been widely accepted 
and opened new windows of opportunity to explore. With its flexibility 
and practicahty, the mobile application can promote learning through playing 
with an interactive environment especially to the children. This paper 
describes the development of mobile learning to help children above 4 years 
old in learning Enghsh and Arabic language in a playful and fun way. 
The application is developed with a combination of Android Studio 
and the machine learning technique, TensorPlow object detection API in 
order to predict the output result. Developed apphcation namely 
“LeamWithIman” has successfuhy been implemented and the results show 
the prediction of apphcation is accurate based on the captured image with 
the hst item. The inclusion of the user database for lesson tracking and new 
lesson will be added for improvement in the future. 

This is an open access article under the CC BY-SA license. 



Corresponding Author: 

Noor Mohd Ariff Brahin, 

Centre for Telecommunication Research & Innovation, 
Universiti Teknikal Malaysia Melaka, 

76100Durian Tunggal, Melaka, Malaysia. 

Email: mohdariff@utem.edu.my 


1. INTRODUCTION 

Mobile devices such as smart phones and tablets are becoming part of human’s daily life 
communication, as weh as entertainment. With the advanced of mobile technologies, it can provide 
an interactive learning content apphcation for children educational purpose which can be simply accessed 
anywhere and anytime. The fact that children are best learns through playing, the making of an interactive 
apphcation for Android mobile attracted the research attention. There is a high potential in the usage 
of mobile technology, in term of how a language can be learned and enhanced. This might be a big help 
for children to become proficient in second language at early age. Stockweh in his paper shows that 
the effects of vocabulary learning through mobile platforms are comparable with the traditional ways [1]. 
This is also supported by Basal et al. through the experiment done that proved the effectiveness of learning 
vocabulary through mobile apphcations compared through the traditional paper-based activities [2]. 
Vocabulary learning is very crucial to build the blocks of language such as read comprehension 
and language speaking [3]. 
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Numerous studies have reported on the use of mobile devices to develop language skills through 
language learning application in past years. Makoe and Shandu implemented a mobile-based application 
to develop vocabulary learning mobile app to enhance the English vocabulary teaching and learning [4]. 
Wang also develop a mobile app to improve the college student’s English vocabulary learning with both 
English and Chinese description [5]. While Sweeney and Moore are investigate the potential of mobile 
devices to be used in language learning. They found that language learning through mobiles has high 
potential but with the coUaborationofbothdeveloper and educator to make sure that it can benefit and help 
the students [6]. The most common method for mobile application development using image processing 
machine learning technique is Mobile Cloud Computing (MCC). MCC is combination of mobiles and cloud 
computing which can simply processing from the user local devices to the data centre facilities over 
the internet [7-24]. The major service providers of MCC are including Google, Amazon, Microsoft 
and Yahoo. Eigure 1 shows the general architecture of MCC. 



Mobile User Internet Data centre - 

cloud storage 


Eigure 1. Mobile cloud computing (MCC) architecture 


The users can simply accessing the cloud storage as long as the internet connection is there. 
However, this can be disadvantage of the MCC. The overall performance of the machine learning can 
be affected depended on the internet connection speed which it cannot perform inference locally 
on the device without internet [25-26]. Thus, TensorElow technique can be a solution to the running machine 
learning on mobile devices. TensorElow is able to runs the machine learning inference computation locally 
on the device. With low latency and small binary size, TensorElow also can be a lot faster and more reliable 
than MCC [27-28]. This paper presents the development of Android mobile application namely 
“LeamWithIman” for language learning through capturing image based on the word given, that aim 
for toddlers above 4 years old to enhance their English and Arabic vocabulary. TensorELow deep learning 
techniques will be used to train the models of captured image for image prediction. 


2. METHOD 

2.1. Android application development 

In order to attract the children interest, “LeamWithIman” is developed to be a fun application with 
the learning through playing concept. “LeamWithIman” introduces two basic languages of English 
and Arabic which divided by different categories. Prior to the application development, the selection of each 
category is crucial to match the level of the children age. The categories are including the ‘Read Easy’, 
‘School Bag’, ‘Sweet Home’, ‘Fruits and Veggies’, ‘Speak Arabic’ and‘Explore Science’. 

Eor the development of the Android mobile application namely “LeamWithIman”, the TensorElow 
deep learning technique and Android Studio were used. As mention in Introduction Section, TensorElow will 
be used as a machine learning method to predict the image captured from “LeamWithIman” application. 
“LeamWithIman” application is preloaded with TensorElow object detection API which uses single shot 
detector (SSD) with MobileNet pre-trained model that is trained on Microsoft Common Object in Context 
(COCO) dataset. The captured image wiU be compared with the preloaded image dataset with the highest 
score in order to predict the output result. Android Studio is used to customize the look of the application 
in Android platform. Eigure 2 provides the architecture of “LeamWithIman” android mobile application. 

Upon entering “LeamWithIman” application, the user is presented with its main interface 
for categories selection as shown in Eigure 3. The application will then display the name of the item with 
sound that requires the user to capture the image of the item. TensorRow will perform the prediction 
by giving the score of the captured image. Once the click button is pressed, the application will compare 
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the score with the name of the item. The correct sign wiUbe displayed if the captured image is matched with 
the name of the item. This will be repeated until aU the items are displayed. The point wiU be shown 
at the end of application. 













Figure 2. “LeamWithIman” application architecture 
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Figure 3. The main interface of “LeamWithIman” application 


The Flowchart of “leamWithIman’ application is shown in Figure 4. While Figure 5 presents 
the screen of image capturing from the application. There are three buttons are available as illustrated in 
Figure 5, left button is the count of correct item based on the captured image, middle button is to capture 
the image from the rear camera and the right button is to skip the current item and go to the next item. 
The name of the item is displayed at the bottom of the application. 
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Eigure 4. The flowchartof“LeamWithIman” android application 
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Eigure 5. Screen of application to capture the image 
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3. RESULTS AND DISCUSSION 

Several experiments have been done with different requirements to observe the practicability 
of the application in image detection and prediction. At first, the application was tested with random 
categories as shown in Table 1. Then, the test for the same item with the different captured image 
was observed to check the reliability of the application. Last but not least, to see the effect of the image 
background for the application prediction. Table 1 presents the comparing scores between the captured image 
and the list item through TensorFlow Object Detection API. As can be seen from the table, “LeamWithIman” 
is able to detect the captured image accurately according to the list item. On average, the detection time 
of the captured image took only 100 ms which considered as sufficient enough. 


Table 1. The score of the matched item from different categories 


Category-Name of the Item 


Read easy-mug 



Detection time (ms) 


105 



Score 


0.58 


0.95 


0.86 


Table 2 illustrates the prediction of the same item but different captured image. It is apparent from 
the table that “LeamWithIman” consistently matches the item correctly even though the image captured 
is different. The results shown indicate that the prediction of the application is reliable. Table 3 provides 
the results of the same item and captured image but with different background. The results obtained revealed 
that “LeamWithIman” able to detect the matched image even with a different background. 
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Table 2. The result from same item with different image 
Captured image Score Prediction 



Table 3. The result of same image with different background 
Captured Image Score Prediction 



0.83 


Matched 


0.69 


Matched 


0.55 


Matched 
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4. CONCLUSION 

This work has presented the development of Android application for children in language learning 
namely “LeamWithIman”. The developed system is able to perform the prediction of the image captured 
by “LeamWithIman” application for children language learning with high accuracy. The application 
is implemented with a combination of interactive elements, visual and sound for pronunciation to attract 
the children interest to learn through mobile application. As for future work, the inclusion of the user 
database for lesson tracking will be considered. Moreover, new lessons for language learning will 
be addedas weU. 
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